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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments on pages 1-3, filed July 20, 2004, with respect to the rejection(s)of 
claim(s) 1 and 31 under 35 U.S.C. § 102(e) have been fully considered and are persuasive. 
Therefore, the rejection has been withdrawn. However, upon further consideration, a new 
ground(s) of rejection is made in view of different interpretation of the previously applied 
reference, Ratakonda (USPN 5,956,026). 

2. Applicant's request for reconsideration of the finality of the rejection of the last Office 
action is persuasive and, therefore, the finality of that action is withdrawn. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

4. Claims 1-5, 11, 12, 31-35, 41, and 42 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Ratakonda (USPN 5,956,026, previously cited). 

5. Regarding claims 1 and 31, Ratakonda discloses a method and computer-readable 
medium containing a program for summarizing a content of an input video sequence, said input 
video sequence comprising a plurality of frames (column 3, line 21-29), said plurality of frames 
being grouped into a plurality of video segments (column 4, line 42-63), said method and 
program comprising: (a) selecting a frame cluster (represented by keyframe kl) in said input 
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video sequence which corresponds to a most static one (minimized action) of said video 
segments (column 6, line 26-column 7, line 33; Figure 6); (b) computing a content value 
(keyframe similarity based on histogram feature vectors) in said selected frame cluster (column 
9, line 13-56); (c) using said computed content value (similarity) to cluster remaining frames 
(keyframes representing video shots) in said input video sequence (column 9, line 44-column 10, 
line 42). 

6. Regarding claims 2 and 32, Ratakonda discloses that in said step (a), said frame cluster is 
selected using a refined feature space representation (treat each histogram as a feature vector of 
its associated frame) of said input video sequence (column 9, line 44-56). 

7. Regarding claims 3 and 33, Ratakonda discloses that in said step (a), each of said 
plurality of frames is transformed into a histogram vector indicative of a spatial distribution of 
colors in said each of said plurality of frames (column 4, line 48-54). 

. 8. Regarding claims 4 and 34, Ratakonda discloses that in said (a) each of said plurality of 
frames (finest level keyframes) is divided into a plurality of blocks, each of said plurality of 
blocks being represented by a histogram in a color space indicative of a distribution of colors 
within each of said plurality of blocks (column 11, line 1-17; column 14, line 15-47). 
9. Regarding claims 5 and 35, Ratakonda discloses that each of said plurality of frames 
(finest level keyframes) is divided into a plurality of blocks and each said histogram vector 
comprises a plurality of histograms in a color space, each of said plurality of histograms 
corresponding to one of said plurality of blocks (column 11, line 1-17; column 14, line 15-47). 
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10. Regarding claims 1 1 and 41, Ratakonda discloses a step (d) including outputting a 
plurality of keyframes, each of said plurality of key frames representative of said clustered 
frames (column 4, line 54-63), 

1 1 . Regarding claims 12 and 42, Ratakonda discloses that said selecting comprises locating a 
cluster (having a histogram vector) closest to an origin (centroid or mean histogram assigned as a 
representative vector) of said refined feature space (column 10, line 6-42). 

Claim Rejections -35 USC § 103 

12. The following is a quotation of 35 U.S. C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

13. Claims 17 and 47 are rejected under 35 U.S.C 103(a) as being unpatentable over 
Ratakonda in view of Lim (USPN 6,574,378, previously cited). 

14. Regarding claims 17 and 47, Ratakonda discloses a method for summarizing a content of 
an input video sequence (column 3, line 21-29), said method comprising: (a) selecting frames 
(keyframes) from said input video sequence (column 4, line 54-63), said selected frames being 
taken at a fixed (equispaced) interval (column 7, line 42-49); (d) selecting a cluster (represented 
by keyframe kl) in said refined feature space corresponding to a most static (minimized action) 
video segment (Figure 6; column 6, line 26-column 7, line 33); (e) computing a content value 
(keyframe similarity based on histogram feature vectors) corresponding to said selected cluster 
(column 9, line 13-56); (f) using said computed content value to cluster frames in said input 
video sequence (column 9, line 44-column 10, line 42). Ratakonda does not disclose (b) creating 
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a feature frame matrix using said selected frames; or (c) performing a singular value 
decomposition on said feature frame matrix to obtain a matrix representing said video sequence 
in a refined feature space. 

15. Lim discloses a method and apparatus for indexing and retrieving images using visual 
keywords wherein during indexing (or retrieval) of a visual document, a spatial aggregation map 
(SAM) of occurrences of visual tokens is created which represents a visual-content signature for 
the visual document (column 9, line 14-28). Subsequently, the SAM is input to a singular-value- 
decomposition (SVD) based coding module to produce a refined feature (reduced 
dimensionality) space representing a coded description of a visual document wherein a frequency 
matrix X associates visual keywords and visual documents by concatenating linearized SAM 
vectors of visual documents as column vectors of the matrix X (column 9, line 29-37). It would 
have been obvious to one of ordinary skill in the art at the time the invention was made to obtain 
a refined feature space using a singular value decomposition of an input video sequence as taught 
by Lim in order to reduce the dimensionality and possibly the noise in the spatial aggregation 
map to produce a coded description of a visual document (column 9, line 29-33). 

16. Claims 6-9 and 36-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ratakonda as applied to claims 2 and 32 above, and further in view of Lim. 

17. Regarding claims 6 and 36, Lim discloses that said refined feature space representation is 
obtained using a singular value decomposition of said input video sequence (column 9, line 14- 
37; see above discussion of claims 17 and 47). 
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18. Regarding claims 7 and 37, Ratakonda discloses that minimization (to a refined feature 
space) is performed using frames (keyframes) selected with a fixed interval (equispaced) from 
said input video sequence (column 7, line 42-49). 

19. Regarding claims 8 and 38, Lim discloses that said selected frames (linearized SAM 
vectors) are arranged into a feature frame matrix (X), and wherein said singular value 
decomposition is performed on said feature frame matrix (column 9, line 33-44). 

20. Regarding claims 9 and 39, Lim discloses that said singular value decomposition 
produces a matrix (X), each column of said matrix representing a frame (SAM vectors of visual 
documents) in a refined feature (reduced dimensionality) space corresponding to a frame in said 
input video sequence (column 7, line 33-44). 

21. Claims 10 and 40 are rejected under 35 U.S.C 103(a) as being unpatentable over 
Ratakonda as applied to claims 1 and 3 1 above, and further in view of Uchihachi et al. (USPN 
6,535,639, previously cited), hereafter Uchichachi. 

22. Regarding claims 10 and 40, Ratakonda does not disclose a step (d) using said clustered 
frames to output a motion video representative of a summary of said input video sequence. 
Uchihachi discloses automatic video summarization using a measure of shot importance and a 
frame-packing method wherein a video may be modified during playback to emphasize the 
measure of shot importance in order to de-emphasize less important shots, or skip shots entirely 
that are less important than a predetermined threshold, resulting in a summary or "video skim" 
(column 9, line 53-60). It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to output a motion video representative of a summary of an input video 
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sequence as taught by Uchihachi in order to emphasize the measure of shot importance by de- 
emphasizing or skipping less important shots and also allowing the user to generate a personal 
video summary (column 9, line 53-60). 

23. Claims 13-16 and 43-46 are rejected under 35 U.S.C 103(a) as being unpatentable over 
Ratakonda as applied to claims 2 and 32 above, and further in view of Castelli et al. (USPN 
6,122,628, newly cited), hereafter Castelli. 

24. Regarding claims 13 and 43, Ratakonda discloses that said step (c) comprises: (c)(1) 
sorting a plurality of vectors in said refined feature space according to a distance of each of said 
vectors to an origin (representative vector) of said refined feature space representation (column 
9, line 49-column 10, line 20); (c)(2) selecting a vector among said sorted vectors which is 
closest to an origin (representative vector) of said refined feature space representation and 
including said selected vector into a first cluster (column 10, line 35-37); (c)(3) clustering said 
plurality of sorted vectors in said refined feature into a plurality of clusters according to a 
distance between each of said plurality of sorted vectors and vectors in each of said plurality of 
clusters and an amount of information in each of said plurality of clusters (column 10, line 37- 
42). Ratakonda does not explicitly disclose that the vectors are sorted in ascending order. 

25. Castelli discloses multidimensional data clustering and dimension reduction for indexing 
and searching wherein eigenvalues (characteristic values of a transformation matrix produced by 
singular value decomposition; column 11, line 9-16) are sorted by decreasing magnitude and a 
subset of ordered eigenvalues containing the largest eigenvalues are selected according to a 
selection criterion (column 11, line 25-31). Castelli sorts in decreasing order, but it would have 
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been obvious to sort in ascending order, depending on the needs of the system. It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to sort a 
plurality of vectors in said refined feature space in ascending order as taught by Castelli in order 
to aid in the selection of the vectors (eigenvalues) according to a user-specified selection 
criterion (column 11, line 28-40). 

26. Regarding claims 14 and 44, Ratakonda discloses that in said step (c)(3) said plurality of 
sorted vectors are clustered into said plurality of clusters such that said amount of information 
(action) in each of said plurality of clusters does not exceed an amount of information (action) in 
said first cluster (column 6, line 45-60), 

27. Regarding claims 1 5 and 45, Ratakonda discloses that said first cluster is composed of 
frames (keyframes) based on a distance variation between said frames (column 8, line 52-63) and 
an average distance between frames in said first cluster (same as the distance since the keyframes 
are equispaced; column 7, line 45-49). 

28. Regarding claims 1 6 and 46, Ratakonda discloses that each of said plurality of clusters is 
composed of frames (keyframes) based on a distance variation between said frames (column 8, 
line 52-63) and an average distance between frames in said each of said plurality of clusters 
(same as the distance since the keyframes are equispaced; column 7, line 45-49). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Ryan J. Hesseltine 
September 21, 2004 



PJ 



